NeuroSense: Short-term emotion recognition and understanding based on spiking neural network modelling of spatio-temporal EEG patterns

Emotion recognition still poses a challenge lying at the core of rapidly growing area affective computing and is crucial for establishing successful human–computer interaction. Identification understanding emotions are achieved through various measures, such as subjective self-reports, face-tracking, voice analysis, gaze-tracking, well analysis autonomic central neurophysiological measurements. Current approaches to emotion based on electroencephalography (EEG) mostly rely handcrafted features extracted over relatively long time windows EEG during participants exposure appropriate stimuli. In this paper, we present short-term framework spiking neural network (SNN) modelling spatio-temporal patterns. Our method relies signal segmentation detection changes in facial landmarks, includes no computation features. Differences between participants’ properties taken into account via subject-dependent spike encoding formulated subject-independent task. We test our methods publicly available DEAP MAHNOB-HCI databases due availability both frontal face video data. Through an exhaustive hyperparameter optimisation strategy, show that proposed SNN-based representation patterns provides valuable information short- term recognition. The obtained accuracies 78.97% 79.39% arousal classification, 67.76% 72.12% valence datasets, respectively. Furthermore, application brain-inspired SNN model, study novel insight helps mechanisms involved emotional processing context audiovisual stimuli, videos. presented results encourage use methodology complement existing commonly used EEG-based recognition, especially